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Melanoma using data augmentation techniques, processing Rol using convolutional 
Skin cancer neural network (CNN) to extract the most important features, and finally the 
Support vector machine extracted CNN features are input to a support vector machine (SVM) 


classifier to decode the two classes benign (B) and malignant (M). Two 
datasets, ISIC and CPTAC-CM, were utilized to train the CNNs. GoogleNet, 
ResNet-50, AlexNet, and VGG19 were investigated and compared. The 
accuracy of the proposed CAD system has reached 99.8% for ISIC database 
and 99.9% for CPTAC-CM database. 
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1. INTRODUCTION 

Skin cancer is characterized by irregular alterations in the skin's outer layer. It is one of the most 
widespread cancers in the world. According to the World Health Organization (WHO), skin cancer has been 
one of the most common forms of cancers and melanoma is one of the leading causes of death, particularly in 
USA, Australia, and New Zealand [1]. While the majority of skin cancer patients are cured, the disease 
remains a significant concern because of its rise pervasiveness [1]. The majority of skin tumors grow only 
internally and penetrate nearby tissues, but others, such as melanoma (pigment cell cancer), which is the most 
extraordinary type of skin malignant growth, can prevalence across the circulatory or lymphatic systems and 
penetrate the most remote parts for the human body [2]. Melanoma has the greatest risk of the various types 
of skin cancer [3]-[7]. Melanoma claimed the lives of 4740 men and 2490 women in 2019 [8]. The findings 
show that diagnosing melanoma in its early stages will dramatically decrease cancer-related mortality. Since 
diagnosing melanoma in an early phase is complex, it will be extremely beneficial to develop a procedure for 
accurate and early detection of melanoma [9]-[12]. The use of image recognition and machine vision to 
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quickly distinguish patterns such as tumors from images decreases human error and improves detection time 
[13]-[19]. One of the most commonly used techniques of image analysis is neural networks. Convolutional 
neural networks (CNNs) represent a form of deep neural networks that are widely used in machine learning 
for medical applications [20]-[25]. An automated decision framework is suggested that involves various 
machine learning techniques to automatically identify melanoma skin lesions with high precision. In the 
present work, both the International Skin Imaging Collaboration (ISIC 2019) dataset [26]-[28] and the 
Clinical Proteomic Tumor Analysis Consortium Cutaneous Melanoma (CPTAC-CM) dataset have been 
utilized [29]. Figures 1 and 2 show examples from the two datasets. Four CNNs are utilized to recognize the 
common nevus from melanoma. To accomplish most elevated accuracy, the framework utilizes the 
accuracies acquired by each method separately. Each process adds a proportion of the final decision taken by 
the main system based on the accuracy obtained. The AlexNet, ResNet-50, GoogleNet and VGG19 were 
fine-tuned by utilizing transfer learning to diagnose skin lesions. In terms of the feature-based process, 
classification will be accomplished using support vector machine (SVM). 

This paper aims to develop a framework for the early prediction of skin cancer utilizing CNN 
modeling, two datasets and data augmentation, to solve the following existing work's limitations: Building 
new CNN models necessitates a massive amount of data for modeling in order to prevent over-fitting. 
Furthermore, these methods are computationally costly, especially during the training process; and Advanced 
techniques must be created and develop new ideas to increase diagnostic accuracy. 

The motivations and contributions of the study will be present as: Select a framework from four 
CNN models to enhance the current skin cancer classification models. This enhancement generates an 
application that appeals to the overall conditions; and propose a very accurate and fast computer-aided 
diagnosis (CAD) system, to leverage deep learning's benefits for image analysis and high performance. 
Because CNN models and frameworks may be retrained using a bespoke dataset for every use case, deep 
learning offers greater flexibility. 

The proposed system has the following contributions: i) The proposed CAD system employs an 
automated optimal approach to isolate the region of interest (RoI) from the background in order to identify 
skin lesions; ii) Investigating and comparing different deep learning architectures for skin cancer detection 
(i.e., AlexNet, ResNet-50, GoogleNet and VGG19); iii) Improving the performance of deep learning training 
using data balancing and augmentation; and iv) Predicting skin cancer early with 100% accuracy. This study 
will aid in the selection of the most appropriate and accurate model for skin cancer diagnosis based on the 
severity of the case critical patients either non-critical patient. 


( "m i el 
(b) (a) (b) 


Figure 1. Examples from ISIC 2019 database [26]- Figure 2. Examples from CPTAC-CM database 
[28], (a) benign and (b) malignant [29], (a) benign and (b) malignant 


2. RELATED RESEARCH 

Over the course of several years, numerous research works on skin cancer have been conducted. 
Some of these techniques have showed substantial prediction accuracy. Many researchers have used image 
segmentation techniques and machine learning classifiers like SVM, KELM, k-nearest neighbor (KNN), and 
FP-ANN to improve treatment for skin cancer detection. 

To detect skin cancer early, Li and Shen [30] introduced a melanoma detection analysis of skin 
lesions. Two deep learning approaches (2 fully convolutional residual networks-—FCRN) were proposed to 
address lesion segmentation, feature extraction, and classification of lesions. The obtained accuracy for lesion 
segmentation was 0.753. Kassem ef al. [31] discussed the papers that have been accepted for publication 
within last five years in ScienceDirect, IEEE, and SpringerLink databases. This research identified the major 
challenges of assessing skin lesion segmentation and classification techniques, such like small datasets, ad 
hoc image selection, and racial bias. Hosny et al. [32], for example, used Alexnet with transfer learning to 
classify the challenging dataset ISIC2018. For lesion segmentation, they used a variety of techniques. The 
obtained accuracy was 92.99%. Hosny et al. [33] used the challenging dataset ISIC2019 and proposed a 
CAD system for skin lesions. That dataset has several flaws, including imbalanced classes and unknown 
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images. The bootstrap weighted classifier was combined utilizing a multiclass SVM by the authors. This 
classifier adjusted the weights based on the image class. The obtained accuracy was 98.70%. Khan et al. [34] 
proposed a fully automated method for the segmentation and categorization of multiclass skin lesions that 
makes use of the most discriminant deep features. They segmented the image utilizing a novel Deep Saliency 
Segmentation technique that employs a ten-layer custom convolutional neural network (CNN). They used the 
kernel extreme learning machine (KELM) classifier to categorise skin lesions. On the ISBI 2016, ISBI 2017, 
ISIC 2018, and PH2 datasets, the accuracies for lesion segmentation were 95.3%, 95.79%, 92.69%, and 
98.70%, respectively. The classification accuracy obtained on the HAM10000 dataset was 90.67%. 
Abayomi-Alli et al. [35] proposed an improved data augmentation model for melanoma skin cancer 
detection. Their method for creating synthetic melanoma images was focused on data oversampling in a 
lower-dimensional nonlinear embedding manifold. The SqueezeNet deep learning model was trained using 
the augmented images. Results in the binary classification problem had shown enhanced melanoma detection 
accuracy of 92.18%, sensitivity of 80.77%, and specificity of 95.1%. The paper is organized as follows; 
section 2 presents detailed descriptions of materials and methods, and section 3 contains experimental results 
and discussion, and finally section 4 gives the conclusions of the paper. 


3. MATERIALS AND METHOD 

The proposed melanoma deep learning framework consists of four processing stages as illustrated in 
Figure 3. First, The Rol is identified from the input image. Second, Raw input data images are augmented 
using ten cropping/flipping, then deep features are extracted from augmented images. Finally, classification 
is carried out in order to establish the skin cancer classes (benign and malignant) grading utilizing a support 
vector machine (SVM) classifier. 
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Figure 3. The proposed CAD scheme for skin cancer detection 


3.1. Collected database and data preparation 

The images utilized in the present work to train and validate the proposed techniques were taken 
from two separate free dermoscopic images datasets; the International Skin Imaging Collaboration (ISIC) 
2019 database [26]-[28], and the clinical proteomic tumor analysis consortium cutaneous melanoma 
(CPTAC-CM) database obtained from the TCIA [29]. The ISIC is an international initiative to enhance early 
detection of melanoma. The ISIC Archive contains about 25,331 JPEG images of skin lesions. The ISIC 
2019 was created specifically for research purposes to aid in the suggestion of new techniques that can be 
used as an assistive diagnostic tool of melanoma [26]-[28]. The TCIA is the archive for cancer imaging and 
associated material at the National Cancer Institute in the United States. It incorporates 30.9 million 
radiology photographs that provides data from 37,568 subjects. Data is divided into categories based on 
tumor type, with several categories containing either analytic observations or clinical evidence [29]. 


3.2. Image segmentation 

One of the most crucial procedures in efficiently classifying lesions is segmenting the lesion from 
the surrounding skin [36]. Because there was a noticeable colour difference between the lesion and the skin, 
thresholding was an excellent choice for this job. As illustrated in Figure 4, a binary image was created with 
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the contrast increased to improve segmentation. To validate the segmentation accuracy, the image is also 
subjected to a boundary tracing algorithm. Thresholding is a straightforward method of segmenting a digital 
image. It can be used with either a greyscale or a colour image. The basic principle underlying thresholding 
involves a parameter called "intensity value" or "brightness threshold," this parameter is selected in such a 
way that given an image I[M,N], if I[M,N] = brightness threshold, consequently I[M,N]=1, else I[M,N]=0. 
This is true if all we care about bright items against a dark background; however, if we are interested in dark 
objects against a light background, the equation would be reversed. The thresholding technique is also a 
computationally low-cost strategy that does not necessitate a large amount of computing time or power. The 
outcomes of segmenting the lesion with the specified algorithm are illustrated in Figure 4. 


Original Image, IM ORG Traced Image Mask, IM mask Image after segmentation 


Figure 4. Extracting roi from the background 


3.3. Data augmentation and balancing 

The ISIC and CPTAC-CM databases are augmented to enhance the efficiency of the proposed deep 
learning framework, enhance the dataset size and prevent over-fitting. To start, the original image and the 
segmented image are resized from their original sizes to 250x250 to allow for high speed processing. We repeat 
the augmentation process described in [37]; five cropped images with dimensions of 224x224 are produced as 
follows: four images are cropped from each corner of the image and one image is cropped around the center of the 
image, so that the size of any cropped image is 224x224. Further, the five cropped images were flipped 
horizontally around the vertical axis to create a total of ten cropped images of size 224x224 as shown in Figure 5. 

Since (i) the number of images per each grade for the ISIC and CPTAC-CM is unbalanced, and (ii) 
the size of the augmented data is ten time the size of the raw data, the proposed system aims to balance the 
data by arbitrarily choosing an equal certain number of images for each grade in order to prevent biasing the 
training results. This step is referred to as "data balancing". The detection rate for each class should ideally be 
near to or the same. We can conclude from the examples above that using a balanced dataset for a model 
results in higher accuracy models, higher precision, and a higher detection rate. 
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Figure 5. Data augmentation process 


3.4. Automatic feature extraction utilizing CNN models 
Previously, specific CNN-driven architectures extracted features that were then merged using a 
dense network into a fully connected layer. CNN models are among the most widely used popular 
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architectures, and they have demonstrated remarkable success in medical applications. These attributes are 
beneficial for classification activities because they reflect various characteristics of features such as visibility, 
roundness of lesion, and shrink. AlexNet, VGG19, ResNet50, and GoogleNet are four successful and recent 
CNN architectures that are being considered as feature extractors for skin cancer classification. Table | provides 
a summary of the pre-trained Resnet-50, Alexnet, Googlenet and VGG19. The next subsections offer a detailed 
overview of certain chosen architectures. Figure 6 shows a block diagram for the proposed method. 
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Figure 6. Workflow of CNN-based systems 


Table 1. Provides a summary of the pre-trained Resnet-50, Alexnet, Googlenet and VGG19 


CNN Model description 
CNN model VP #Layers #CL _#FC _Top-l/top-5 Error(%) 
AlexNet 227x227 8 5 3 41.00/18.00 
ResNet-50 224x224 50 49 1 22.85/6.71 
GoogleNet 224x224 22 21 1 29.81/10.04 
VGG19 224x224 19 16 3 27.30/9.00 


In Table 1, the symbol "#" denotes the number, I/P denotes the input image size, FC denotes fully 
connected, and CL denotes convolutional layers. AlexNet is composed of three fully linked layers and five 
convolutional layers. More information about AlexNet and can be found in [38]. Figure 7 shows the complex 
architecture of AlexNet. 

ResNet is an abbreviation for Residual Network. ResNet-50 is a network that addresses the issue of 
accuracy degradation as network depth increases, allowing accuracy to become saturated and quickly 
degrading. It's a more robust 50 layers network that can identify 1000 objects in a single iteration. This 
network avoids the vanishing gradient issue that plagues other deep networks, making the associated features 
and classification challenge difficult to optimize. The implementation of skip net connections, where input is 
often applied to the outcome of the network's convolution blocks, is the power of ResNet-50 [39]. Figure 8 
shows the complex architecture of ResNet-50. 

VGG19 is similar to AlexNet but it also has more convolutional layers for image recognition. 
VGG19 is a deep CNN model with 19 layers with three linked layers at different levels and a pooling layer. 
This model has historically played a key role in image classification and localization, thanks to the help of 
bounding box architectures. It employs a convolutional filter with a size of 3x3 and a pooling layer network 
with a size of 2x2. Figure 9 depicts the complex architecture of VGG19. 
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Figure 7. AlexNet architecture 
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Figure 8. ResNet-50 architecture 
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Figure 9. The personalized VGG-19 network architecture 
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GoogleNet is a convolutional neural network that is 22 layers deep. It was developed by Google 
researchers and is known as Inception Network. It's capable of pooling or convolving input data. As seen in 
Figure 10, the architecture consists of several inception modules stacked on top of one another. This stacking 
enables joint and parallel training, allowing the model to achieve quicker convergence. The benefits include 


quicker training and smaller scale. It does, however, have an Xception Network that will enhance the 
inception module's limit of divergence [40]. 
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Figure 10. The customized GoogleNet network architecture 


The weights of the pretrained models' convolutional layers are transferred without training to 
minimize training overhead, and then the completely linked layers are trained using the two datasets. 
Transfer learning is applied to CNN models using the final fully linked layer of the pre-trained models. 
The parameters of the introduced layers are trained using our ISIC and CPTAC-CM data. The vectors of 
activities of the last fully connected layer just before FC8 layer in AlexNet and VGG19 (i.e, the FC7 layer) 
or just before FC1000 in Res-Net50 or just before pool5-7x7_s1 in GoogleNet (i.e., the flatten layer), 
constitute the two datasets’ feature descriptors. Figure 11 summarizes the proposed method. 
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Figure 11. Block diagram of the proposed method 


3.5. Classification 

An SVM shallow classifier is utilized to categorise dermoscopic images. By breaking hyper-planes 
in a high-dimensional feature space, SVM aims to build a computationally efficient learning system [41]. The 
Rol images are classified as B or M in this phase depending on the deep extracted features. The classifier is 
fed with activity vectors (FC7 in AlexNet and VGG19, or the flatten layer in ResNet50, or pool5-7x7 s1 in 
GoogleNet), and the RoI class is output. To represent the variability of the classes, the SVM uses multi-linear 
mapping with a simple binary kernel. The performance is assessed for either the ResNet50 or AlexNet or 
VGG19 or GoogleNet, each along with either NN classifier or the SVM classifiers are dependent on standard 
performance evaluation metrics, such as, the classification precision must be considered, in order to choose 
the right model/classifier. 


3.6. Performance evaluation 
Three standard parameters are utilized to measure the efficiency of the suggested technique; 
accuracy (ACC.), sensitivity (SEN.), and specificity (SPE.). The evaluation metrics are described as [42]: 


No. of correct assessments TP+TN 


ACC. = = ——____. (1) 
No. of assessments TP+FN+FP+TN 
No. of true positive assessments TP 
SEN. = = = (2) 
No. of positive assessments TP+FN 
No. of true negative assessments TN 
SPE. = : = (3) 
No. of negative assessments TN+FP 


where: 

TP-true positive: a malignant image is correctly identified as malignant. 
TN-true negative: a benign image is correctly identified as benign. 
FP-false positive: a benign image is mistakenly diagnosed as cancerous. 
FN-false negative: a malignant image is mistakenly classified as benign. 


4. EXPERIMENTAL RESULTS AND DISCUSSION 

The proposed system sensitivity, specificity and accuracy for using the SVM classifiers for skin 
cancer system have been determined. Two databases are used to test the proposed method. Each dataset is 
divided into two parts: training set and test set. The following section describes the experimental 
configuration, findings, and related discussions in detail. 


4.1. Experiment settings 

The system classifier is trained on 70% of the dataset and tested on 30% of the database. The 
Bayesian optimizer is employed in order to train the deep learning models (AlexNet, VGG19, ResNet50, and 
GoogleNet), by minimizing the binary cross entropy function with a learning rate of 10* . A mini-patch size 
of 128 is utilised to shuffle the data through preparation. The maximum number of epochs that can be utilised 
is 20. The configuration values are listed in Table 2. 
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Table 2. The configuration values 


Configuration Value 
Learning rate 104 
Mini-patch size 128 
Number of epochs 20 


Maximum iterations _ 606 (in case of alexnet), 642 (in case of resnet), 612(in case of googlenet), 600(in case of VGG19) 


4.2. Quantitative results 

The behavior of four CNNs as individual classifiers for melanoma identification is investigated. The 
best results for AlexNet and ResNet-50 CNNs, pre-trained and adapted to all datasets, are shown in 
Figures 12 and 13. The weight learning rate factor and the bias learn rate factor were both set to 20 in this 
case, with a tiny batch size of 1. 
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Figure 13. Training progress using ResNet-50 CNN for TCIA 


As a result, some samples of validation images are shown in Figure 14 for ISIC and CPTAC-CM 
databases respectively. We provided the predicted labels as well as the predicted probability of the images 
containing certain labels for each of them. The validation accuracies were 99.3% and 99.6%, respectively, in 
case of ResNet-50. In the global decision-fusion system, this precision would be the related weight for the 
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AlexNet classifier. Figures 15-18 show the confusion matrix to categorise the two class labels utilizing CNNs 
models and balanced data utilizing the suggested method. As seen in the figures, the overall accuracy of all 
classes has reached 99.8%, which, as expected, outperforms the case of utilising imbalanced data. The 
receiver operating characteristic (ROC) curve is utilized to display the results of the proposed simulation 
model. Figures 19 and 20 show the ROC curves for categorizing the 2 classes utilizing AlexNet and ResNet- 
50 CNNs and balanced data utilizing our proposed technique. Table 3 gives the results of the proposed 
system. Table 4 gives a comparison between the proposed approach and the previously suggested techniques 
in a brief manner. 


C] Correct C] Incorrect C] Correct oe 
estimation estimation estimation estimation 
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Figure 14. Validation images with foretold labels in the case of ResNet-50 for (a) CPTAC-CM datasets and 
(b) ISIC datasets 
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Figure 17. Confusion Matrix presenting 
categorization results in case of AlexNet for 
ISIC dataset 


Figure 18. Confusion Matrix presenting 
categorization results in case of ResNet-50 for 
ISIC dataset 
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Figure 19. ROC curves in skin lesions classification 
of ISIC 2019 (Plot of TPR vs FPR for two distinct 
diagnostic categories: Melanoma, Non-Melanoma) 
AlexNet CNN for ISIC 


Figure 20. ROC curves in skin lesions classification 
of ISIC 2019 (Plot of TPR vs FPR for two distinct 
diagnostic categories: Melanoma, Non-Melanoma) 

ResNet-50 CNN for ISIC 


4.3. Discussion 

Table 3 shows the accuracies, sensitivities, and specificities of the different training data setting of 
the proposed system. Where proposed method 1 corresponds to ISIC dataset with data augmentation, and 
proposed method 2 corresponds to CPTAC-CM dataset with data augmentation. We utilized CNNs that had 
been pre-trained with a huge image dataset to detect skin lesions. The previously explained four models are 
applied and compared for each case. As shown Table 3 demonstrates that using AlexNet and GoogleNet with 
a SVM classifier achieves the highest performance in all examined cases. These results highlight the 
potentiality of using AlexNet/SVM and GoogleNet/SVM, taking into account augmentation, for achieving 
high performance, the mean accuracies are 99.9% and 99.8% respectively. Utilizing transfer learning, the 
CNNs were able to classify skin lesions successfully. The best findings were achieved utilizing AlexNet and 
GoogleNet pre-trained with the two datasets. In comparison with previous works, shown in Table 4, the 
presented model had proven high accuracy in early detection of skin cancer. This demonstrates the 
approach's capability to detect melanoma at an early stage. When comparing the findings of several research, 
as shown in Table 4, to the results of the present study, our technique performed even better by using 4 deep 
neural networks. 
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Table 3. Proposed system for different training data using four deep learning models (Resnet50, Alexnet, 
Googlenet And Vgg19) With SVM Classifier and transfer learning classifier and using two datasets 


SVM Classifier Transfer Learning 
Case Dataset Method ACC. % SEN.% SPE.% ACC.% SEN.% SPE. % 
Case I ISIC ResNet50 99.3% 99.5% 99.1% 99.5% 99.5% 99.6% 
CPTAC-CM  ResNet50 99.6% 99% 100% 100% 100% 100% 
Case ISIC VGG19 98.8% 98.5% 99.1% 98.9% 98.6% 99.2% 
CPTAC-CM VGG19 99.6% 100% 99.3% 99.8% 99.5% 100% 
Case IV ISIC GoogleNet 99.8% 99.5% 100% 99.6% 99.8% 100% 
CPTAC-CM GoogleNet 99.87% 99.5% 100% 99.9 % 100% 100% 
Case V ISIC AlexNet 98.8% 99% 98.7% 99.31% 99.54% 99.05% 


CPTAC-CM _ AlexNet 99.9% 100% 99.6% 100% 100% 100% 


Table 4. Comparison between related methods and proposed models 


References Dataset Method Observations Accuracy % 

Li and Shen [30] ISIC Two deep learning method lesion classification 91.2% 

Hosny et al. [32] ISIC2018 AlexNet, 10-fold cross-validation, fine-tune, lesion classification 92.99% 
transfer learning, Augmentation, GPU 

Hosny et al. [33] ISIC2019 GoogleNet, Similarity score, bootstrap weighted lesion classification 98.70% 


SVM classifier, SoftMax, fine-tune, transfer 
learning, Augmentation. 


Khan et al. [34] ISBI2016 Novel Deep Saliency Segmentation method, ten- lesion classification 95.38% 

ISBI 2017 layer convolutional neural network (CNN), and 95.79% 

ISBI 2018 Kernel Extreme Learning Machine (KELM) 92.69% 

PH2 classifier. 98.70% 

Abayomi-Alli PH2 Deep CNN-based using SqueezeNet and lesion classification 92.18% 
et al. [35] covariant SMOTE augmentation in low 

dimensional manifold. 

Kawahara et al. [43] ISIC Linear classifier CNN deep learning. lesion classification 85.5% 

Codella et al. [44] ISIC Deep learning, sparse coding, SVM. lesion classification 93.1% 

Proposed Method 1 ISIC CNNs based on GoogleNet. lesion classification 99.8% 

CNNs based on AlexNet. lesion classification 98.8% 

CNNs based on ResNet-50. lesion classification 99.3% 

CNNs based on VGG19. lesion classification 98.8% 

Proposed Method 2 CPTAC- CNNs based on GoogleNet. lesion classification 99.87 % 

CM CNNs based on AlexNet. lesion classification 99.9% 

CNNs based on ResNet-50. lesion classification 99.6% 

CNNs based on VGG19. lesion classification 99.6% 


4.3.1. Limitations 

The color and texture of different types of skin lesions differ only slightly. Diverse images may 
result in images of skin lesions that do not belong to that specific lesion category. Another issue was the 
existence of certain dermatologist-drawn marks close to the lesions, as well as the existence of drops of 
water. Despite producing promising results in comparison to competing techniques, the proposed method 
fails to correctly classify all of the images. There are some images in Figure 14 that depict examples of 
erroneous classification. In the future, it is intended to add more features and different deep learning 
algorithms to enhance the accuracy of classification. 


5. CONCLUSION 

In this paper, a CAD system for early detection of skin cancer was presented based on deep learning. The 
CNN models utilized extracted features from Rol in order to enhance the deep learning model training capabilities. 
A SVM classifier was utilized to categorise dermoscopic images into two classes; benign or malignant. 
Experimental results using four CNN models on two different datasets had confirmed the superiority of the 
proposed methods over previous work. Results have shown that, under all conditions, the AlexNet and ResNet-50 
frameworks were superior to other models with an average accuracy of 99.9% and 100% respectively. These 
results confirm that the proposed technique can be utilized as an effective diagnostic tool for early diagnosis of 
melanoma. Future work will focus on multi-classification and grading of different types of skin cancer. Another 
challenge is to reduce computational time and to use different datasets with larger number of images. 
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